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(57) Abstract 

A method for controlling a computer (302) by inputting an analog signal (111) into the computer (302) to control a web browser 
software application. The analof signal (111) contains a trigger signal which activates proprietary software, and a product 
identifier. The proprietary sofware launches the web browser application on the computer (302), extracts the product identifier, 
and creates an appended data string by appending server address (URL) routing information to the product identifier information. 
The appended data string is automatically inserted into the web browser as keystroke data and routed to an advertiser reference 
server (308). The appended routing information directs communication to the adverstiser reference server (308) which contains a 
cross-referenced database (310) of advertiser product identifier information and associated advertiser server URLs. The 
advertiser server URL and a request for product information relevant to the product identifier is returned to the computer (302) 
web browser where it is automatically redirected to the advertiser server (312) containing the advertiser product information. 
The advertiser product information is then returned to the computer (302) for display. 

(57) Abrege 

La presente invention concerne un procede de commande d'un ordinateur (302) par entree d'un signal analogique (111) dans 
I'ordinateur (302) afin de commander une application de programme de navigateur Web. Le signal analogique (111) contient un 
signal declencheur qui active le logiciel brevete et un identificateur de produit. Le logiciel brevete lance, sur I'ordinateur 
(302), I'application de navigateur Web, extrait I'identificateur de produit, et cree une chaine de donnees jointes en ajoutant 
une information d'acheminement d'adresse de serveur (URL) a I'information d'identificateur de produit. Cette chaine de donnees 
jointes est automatiquement inseree dans le navigateur Web comme donnees de frappe et acheminee a un serveur de reference 
d'annonceur (308). L'information d'acheminement jointe dirige une communication vers le serveur de reference d'annonceur (308) 
qui contient une base de donnees d'equivalences (310) d'information d'identificateur de produit d'annonceur et d'adresses URL de 
serveur d'annonceur associe. L'adresse URL de serveur d'annonceur et une demande d'information de produit relative a 
I'identificateur de produit est renvoyee au navigateur Web de I'ordinateur (302), ou elle est automatiquement dirigee de nouveau 
vers le serveur d'annonceur (312) qui contient l'information de produit d'annonceur. L'information de produit d'annonceur est 
alors renvoyee a I'ordinateur (302) pour affichage. 
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information directs communication to the adverstiser reference server (308) which contains a cross-referenced database (310) of advertiser 
product identifier information and associated advertiser server URLs. The advertiser server URL and a request for product information 
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METHOD FOR CONTROLLING A COMPUTER 
WITH AN AUDIO SIGNAL 



TECHNICAL FIELD OF THE INVENTION 

This invention is related to a method of computer control, and particularly for 
automatically directing a web browser application on the computer to retrieve and display 
information in response to an analog signal. 



CROSS REFERENCE TO RELATED APPLICATION 

This application is related to copending U.S. Patent Application Serial No. 09/151,471, 
entitled, "METHOD FOR INTERFACING SCANNED PRODUCT INFORMATION WITH A 
SOURCE FOR TFIE PRODUCT OVER A GLOBAL NETWORK" (Arty DktNo. PHLY- 
24,397) filed of even date herewith. 
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BACKGROUND OF THE INVENTION 

With the growing numbers of computer users connecting to the "Internet,*' many 
companies are seeking the substantial commercial opportunities presented by such a large user 
base. For example, one technology which exists allows a television ("TV") signal to trigger a _ 
computer response in which the consumer will be guided to a personalized web page. The source 
of the triggering signal may be a TV, video tape recorder, or radio. For example, if a viewer is 
watching a TV program in which an advertiser offers viewer voting, the advertiser may transmit a 
unique signal within the television signal which controls a program known as a "browser" on the 
viewer's computer to automatically display the advertiser's web page. The viewer then simply 
makes a selection which is then transmitted back to the advertiser. 

In order to provide the viewer with the capability of responding to a wide variety of 
companies using this technology, a database of company information and Uniform Resource 
Locator ("URL") codes is necessarily maintained in the viewer's computer, requiring continuous 
updates. URLs are short strings of data that identify resources on the Internet: documents, 
images, downloadable files, services, electronic mailboxes, and other resources. URLs make 
resources available under a variety of naming schemes and access methods such as HTTP, FTP, 
and Internet mail, addressable in the same simple way. URLs reduce the tedium of "login to this 
server, then issue this magic command ..." down to a single click. The Internet uses URLs to 
specify the location of files on other servers. A URL includes the type of resource being 
accessed (e.g., Web, gopher, FTP), the address of the server, and the location of the file. The 
URL can point to any file on any networked computer. Current technology requires the viewer to 
perform periodic updates to obtain the most current URL database. This aspect of the current 
technology is cumbersome since the update process requires downloading information to the 
viewer's computer. Moreover, the likelihood for error in performing the update, and the 
necessity of redoing the update in the event of a later computer crash, further complicates the 
process. Additionally, current technologies are limited in the number of companies which may 
be stored in the database. This is a significant limitation since world-wide access presented by 
the Internet and the increasing number of companies connecting to perform on-line commerce 
necessitates a large database. 
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SUMMARY OF THE INVENTION 

The present invention disclosed and claimed herein comprises a method for retrieving 
information from a storage region having a defined location. A program is broadcasted having 
embedded therein a routing signal having routing information contained therein. The routing 
signal is then extracted from the broadcast. Thereafter, a personal comp uteris controlled to allow 
a user to retrieve the information from a storage region at the defined location, which defined 
location is located with the extracted routing information, providing it at the personal computer 
for use by the user. 
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For a more complete understanding of the present invention and the advantages thereof, 
reference is now made to the following description taken in conjunction with the accompanying 
Drawings in which: 

FIGURE 1 illustrates a block diagram of the preferred embodiment; 
FIGURE 2 illustrates the computer components employed in this embodiment; 
FIGURE 3 illustrates system interactions over a global network; 

FIGURES 4a-4e illustrate the various message packets transmitted between the source PC 
and network servers used in the preferred embodiment; and 

FIGURE 5 is a flowchart depicting operation of the system according to the preferred 
embodiment. 

FIGURE 6 illustrates a flowchart of actions taken by the Advertiser Reference Server 
("ARS") server; 

FIGURE 7 illustrates a flowchart of the interactive process between the source computer 
and ARS; 

FIGURE 8 illustrates a web browser page receiving the modified URL/advertiser product 
data according to the preferred embodiment; 

FIGURE 9 illustrates a simplified block diagram of the disclosed embodiment; 

FIGURE 10 illustrates a more detailed, simplified block diagram of the embodiment of 
FIGURE 9; 

FIGURE 1 1 illustrates a diagrammatic view of a method for performing the routing 
operation; 

FIGURE 12 illustrates a block diagram of an alternate embodiment utilizing an optical 
region in the video image for generating the routing information; 

FIGURE 13 illustrates a block diagram illustrating the generation of a profile with the 
disclosed embodiment; 

FIGURE 14 illustrates a flowchart for generating the profile and storing at the ARS; and 

FIGURE 15 illustrates a flowchart for processing the profile information when 
information is routed to a user. 
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Referring now to FIGURE 1, there is illustrated a block diagram of a system for 
controlling a personal computer ("PC") 112 via an audio tone transmitted over a wireless system 
utilizing a TV. In the embodiment illustrated in FIGURE 1, there is provided a transmission 
station 101 and a receive station 1 1 7 that are connected via a communication link 1 08. The 
transmission station 101 is comprised of a television program source 104, which is operable to 
generate a program in the form of a broadcast signal comprised of video and audio. This is 
transmitted via conventional techniques along channels in the appropriate frequencies. The 
program source is input to a mixing device 106, which mixing device is operable to mix in an 
audio signal. This audio signal is derived from an audio source 100 which comprises a coded 
audio signal which is then modulated onto a carrier which is combined with the television 
program source 104. This signal combining can be done at the audio level, or it can even be done 
at the RF level in the from of a different carrier. However, the preferred method is to merely sum 
the audio signal from the modulator 102 into the audio channel of the program that is generated 
by the television program source 104. The output thereof is provided from the mixing device 106 
in the form of broadcast signal to an antenna 107 , which transmits the information over the 
communication link 108 to an antenna 109 on the receive side. 

On the receive side of the system, a conventional receiver 110, such as a television is 
provided. This television provides a speaker output which provides the user with an audible 
signal. This is typically associated with the program. However, the receiver 1 10 in the disclosed 
embodiment, also provides an audio output jack, this being the type RCA jack. This jack is 
utilized to provide an audio output signal on a line 1 13 which is represented by an audio signal 
111. This line 1 13 provides all of the audio that is received over the communication link 108 to 
the PC 1 12 in the audio input port on the PC 1 12. However, it should be understood that, 
although a direct connection is illustrated from the receiver 1 10 to the PC 112, there actually 
could be a microphone pickup at the PC 112 which could pick the audio signal up. In the 
disclosed embodiment the audio signal generated by the advertiser data input device 100 is 
audible to the human ear and, therefore, can be heard by the user. Therefore, no special fdters are 
needed to provide this audio to the PC 112. 
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The PC 1 12 is operable to run programs thereon which typically are stored in a program 
file area 116. These programs can be any type of programs such as word processing programs, 
application programs, etc. In the disclosed embodiment, the program that is utilized in the 
system is what is referred to as a "browser." The PC 1 12 runs a browser program to facilitate 
the access of information on the network, for example, a global communication network known _ 
as the "Internet" or the World-Wide- Web ("Web")- The browser is a hypertext-linked 
application used for accessing information. Hypertext is a term used to describe a particular 
organization of infonnation within a data processing system, and its presentation to a user. It 
exploits the computer's ability to link together information from a wide variety of sources to 
provide the user with the ability to explore a particular topic. The traditional style of presentation 
used in books employs an organization of the information which is imposed upon it by 
limitations of the medium, namely fixed sized, sequential paper pages. Hypertext systems, 
however, use a large number of units of text or other types of data such as image information, 
graphical information, video information, or sound information, which can vary in size. A 
collection of such units of information is termed a hypertext document, or where the hypertext 
documents employ information other than text, hypermedia documents. Multimedia 
communications may use the Hypertext Transfer Protocol ("HTTP"), and files or formatted data 
may use the Hypertext Markup Language ("HTML"). This formatting language provides for a 
mingling of text, graphics, sound, video, and hypertext links by "tagging" a text document using 
HTML. Data encoded using HTML is often referred to as an "HTML document," an "HTML 
page," or a "home page." These documents and other Internet resources may be accessed across 
the network by means of a network addressing scheme which uses a locator referred to as a 
Uniform Resource Locator ("URL"), for example, "http://www.digital.com." 

The Internet is one of the most utilized networks for interconnecting distributed computer 
systems and allows users of these computer systems to exchange data all over the world. 
Connected to the Internet are many private networks, for example, corporate or commercial 
networks. Standard protocols, such as the Transport Control Protocol ("TCP") and the Internet 
Protocol ("IP") provide a convenient method for communicating across these diverse networks. 
These protocols dictate how data are formatted and communicated. As a characteristic of the 
Internet, the protocols are layered in an IP stack. At higher levels of the IP stack, such as the 
application layer (where HTTP is employed), the user information is more readily visible, while 
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at lower levels, such as the network level (where TCP/IP are used), the data can merely be 
observed as packets or a stream of rapidly moving digital signals. Superimposed on the Internet 
is a standard protocol interface for accessing Web resources, such servers, files, Web pages, mail 
messages, and the like. One way that Web resources can be accessed is by browsers made by 
Netscape® and Microsoft Internet Explorer®. 

Referring again now to FIGURE 1, the user can load this program with the appropriate 
keystrokes such that a browser window will be displayed on a display 118. In one embodiment, 
the user can run the browser program on the PC 1 12 such that the browser window is displayed 
on the display 118. While watching a preferred program, the user can also view display 118. 
When an audio signal is received by the receiver 1 10 and the encoded information is contained 
therein that was input thereto by the advertiser, the PC 1 12 will then perform a number of 
operations. The first operation, according to the disclosed embodiment, is to extract the audio 
information within the received audio signal in the form of digital data, and then transmit this 
digital data to a defined location on the global communication network via a modem connection 
1 14. This connection will be described hereinbelow. This information will be relayed to a 
proprietary location and the instructions sent back to the PC 1 12 as to the location of the 
advertiser associated with the code, and the PC 1 12 will then effect a communication link to that 
location such that the user can view on the display 1 1 8 information that the advertiser, by the fact 
of putting the tone onto the broadcast channel, desires the viewer to view. This information can 
be in the form of interactive programs, data files, etc. In one example, when an advertisement 
appears on the television, the tone can be generated and then additional data displayed on the 
display 118. Additionally, a streaming video program could be played on the PC received over 
the network, which streaming video program is actually longer than the advertising segment on 
the broadcast. Another example would be a sports game that would broadcast the tone in order to 
allow a user access to information that is not available over the broadcast network, such as 
additional statistics associated with the sports program, etc. 

By utilizing the system described herein with respect to the disclosed embodiment of 
FIGURE 1, an advertiser is allowed the ability to control a user's PC 1 12 through the use of tones 
embedded within a program audio signal. As will descried hereinbelow, the disclosed 
embodiment utilizes particular routing information stored in the PC 1 12 which allows the 
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encoded information in the received audio signal to route this information to a desired location on 
the network and then allow other routing information to be returned to the PC 1 12 for control 
thereof to route the PC 1 12 to the appropriate location associated with that code. 

Referring now to FIGURE 2, there is illustrated a computer 204, similar to computer 1 12 A 
connected to display information on display 1 1 8. The computer 204 comprises an internal audio 
or "sound" card 206 for receiving the transmitted audio signal through receive antenna 109 and 
receiver 110. The sound card 206 typically contains analog-to-digital circuitry for converting the 
analog audio signal into a digital signal. The digital signal may then be more easily manipulated 
by software programs. The receiver 110 separates the audio signal from the video signal. A 
special trigger signal located within the transmitted advertiser audio signal triggers proprietary 
software running on the computer 204 which launches a communication application, in this 
particular embodiment, the web browser application located on the PC 204. Coded advertiser 
information contained within the audio signal is then extracted and appended with the address of 
a proprietary server located on the communication network. The remote server address is in the 
form of a URL. This appended data, in addition to other control codes, is inserted directly into 
the web browser application for automatic routing to the communication network. The web 
browser running on PC 204, and communicating to the network with a through an internal 
modem 208, in this embodiment, transmits the advertiser information to the remote server. The 
remote server cross-references the advertiser product information to the address of the advertiser 
server located on the network. The address of the advertiser server is routed back through the PC 
204 web browser to the advertiser server. The advertiser product information is returned to PC 
204 to be presented to the viewer on display 118. In this particular embodiment, the particular 
advertiser product information displayed is contained within the advertiser's web page 212. As 
mentioned above, the audio signal is audible to the human ear. Therefore the audio signal, as 
emitted from the TV speakers, may be input to the sound card 206 via a microphone. 
Furthermore, the audio signal need not be a real-time broadcast, but may be on video tapes, CDs, 
DVD, or other media which may be displayed at a later date. With the imminent implementation 
of high definition digital television, the audio signal output from the TV may also be digital. 
Therefore, direct input into a sound card for AID purposes may not be necessary, but alternative 
interfacing techniques to accommodate digital-to-digital signal formats would apply. 



WO 00/1 6205 PCT/US99/20963 

9 

Referring now to FIGURE 3, there is illustrated a source PC 302, similar to PCs 204 and 
112, connected to a global communication network 306 through an interface 304. In this 
embodiment, the audio signal 1 1 1 is received by PC 302 through its sound card 206. The audio 
signal 1 1 1 comprises a trigger signal which triggers proprietary software into launching a web 
browser application residing on the PC 302. The audio signal 1 1 1 also comprises advertiser 
product information which is extracted and appended with URL information of an Advertiser 
Reference Server ("ARS") 308. The ARS 308 is a system disposed on the network that is 
defined as the location to which data in the audio signal 1 1 1 is to be routed. As such, data in the 
audio signal 1 1 1 will always be routed to the ARS 308, since a URL is unique on the network 
system. Connected to the ARS 308 is a database 310 of product codes and associated 
manufacturer URLs. The database 310 undergoes a continual update process which is 
transparent to the user. As companies sign -on, e.g., subscribe, to this technology, manufacturer 
and product information is added to the database 310 without interrupting operation of the source 
PC 302 with frequent updates. When the advertiser server address URL is obtained from the 
ARS database 310, it and the request for the particular advertiser product information is 
automatically routed back through the web browser on PC 302, over to the respective advertiser 
server for retrieval of the advertiser product information to the PC 302. It should be noted that 
although the disclosed invention discusses a global communication network, the system is also 
applicable to LANs, WANs, and peer-to-peer network configurations. It should be noted that the 
disclosed architecture is not limited to a single source PC 302, but may comprise a plurality of 
source PCs, e.g., PC 300 and PC 303. Moreover, a plurality of ARS 308 systems and advertiser 
servers 312 may be implemented, e.g., ARS 314, and advertiser server A 316, respectively. 

The information transactions, in general, which occur between the networked systems of 
this embodiment, over the communication network, are the following. The web browser running 
on source PC 302 transmits a message packet to the ARS 308 over Path "A." The ARS 308 
decodes the message packet and performs a cross-reference function with product information 
extracted from the received message packet to obtain the address of an advertiser server 3 12. A 
new message packet is assembled comprising the advertiser server 3 1 2 address, and sent back to 
the source PC 302 over Path "B." A "handoff ' operation is performed whereby the source PC 
302 browser simply reroutes the information on to the advertiser server 312 over Path "C," with 
the appropriate source and destination address appended. The advertiser server 3 12 receives and 
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decodes the message packet. The request- for-advertiser-product-infomiation is extracted and the 
advertiser 312 retrieves the requested information from its database for transmission back to the 
source PC 302 over Path "D " The source PC 302 then processes the information, i.e., for 
display to the viewer. The optional Path "E" is discussed hereinbelow. It should be noted that 
the disclosed methods are not limited to only browser communication applications, but may 
accommodate, with sufficient modifications by one skilled in the art, other communication 
applications used to transmit information over the Internet or communication network. 

Referring now to FIGURE 4a, the message packet 400 sent from the source PC 302 to 
ARS 308 via Path "A" comprises several fields. One field comprises the URL of the ARS 308 
which indicates where the message packet is to be sent. Another field comprises the advertiser 
product code or other information derived from the audio signal 111, and any additional overhead 
information required for a given transaction. The product code provides a link to the address of 
the advertiser server 312, located in the database 3 1 0. Yet another field comprises the network 
address of the source PC 302. In general, network transmissions are effected in packets of 
information, each packet providing a destination address, a source address, and data. These 
packets vary depending upon the network transmission protocol utilized for communication. 
Although the protocols utilized in the disclosed embodiment are of a conventional protocol suite 
commonly known as TCP/IP, it should be understood that any protocols providing the similar 
basic functions can be used, with the primary requirement that a browser can forward the routing 
information to the desired URL in response to keystrokes being input to a PC. However, it 
should be understood that any protocol can be used, with the primary requirement that a browser 
can forward the product information to the desired URL in response to keystrokes being input to 
a PC. Within the context of this disclosure, "message packet" shall refer to and comprise the 
destination URL, product information, and source address, even though more than a single packet 
must be transmitted to effect such a transmission. 

Upon receipt of the message packet 400 from source PC 302, ARS 308 processes the 
information in accordance with instructions embedded in the overhead information. The ARS 
308 specifically will extract the product code information from the received packet 400 and, once 
extracted, will then decode this product code information. Once decoded, this information is then 
compared with data contained within the ARS advertiser database 310 to determine if there is a 
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"hit." If there is no "hit" indicating a match, then information is returned to the browser 
indicating such. If there is a "hit," a packet 402 is assembled which comprises the address of the 
source PC 302, and information instructing the source PC 302 as to how to access, directly in a 
"handoff 1 operation, another location on the network, that of an advertiser server 312. This type 
of construction is relatively conventional with browsers such as Netscape® and Microsoft 
Internet Explorer® and, rather than displaying information from the ARS 308, the source PC 302 
can then access the advertiser server 312. The ARS 308 transmits the packet 402 back to source 
PC 302 over Path "B." Referring now to FIGURE 4b, the message packet 402 comprises the 
address of the source PC 302, the URL of the advertiser server 312 embedded within 
instructional code, and the URL of the ARS 308. 

Upon receipt of the message packet 402 by the source PC 302, the message packet 402 is 
disassembled to obtain pertinent routing information for assembly of a new message packet 404. 
The web browser running on source PC 302 is now directed to obtain, over Path "C" the product 
information relevant to the particular advertiser server 312 location information embedded in 
message packet 404. Referring now to FIGURE 4c, the message packet 404 for this transaction 
comprises the URL of the advertiser server 312, the request-for-pro duct-information data, and the 
address of the source PC 302. 

Upon receipt of the message packet 404 from source PC 302, advertiser server 312 
disassembles the message packet 404 to obtain the request-for-product-information data. The 
advertiser server 312 then retrieves the particular product information from its database, and 
transmits it over Path "D" back to the source PC 302. Referring now to FIGURE 4d, the 
message packet 406 for this particular transaction comprises the address of the source PC 302, 
the requested information, and the URL of the advertiser server 312. 

Optionally, the ARS 308 may make a direct request for product information over Path 
"E" to advertiser server 312. In this mode, the ARS 308 sends information to the advertiser 
server 312 instructing it to contact the source PC 302. This, however, is unconventional and 
requires more complex software control. The message packet 408 for this transaction is 
illustrated in FIGURE 4e, which comprises the URL of the advertiser server 3 12, the request-for- 
product-information data, and the address of the source PC 302. Since product information is not 
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being returned to the ARS 308, but directly to the source PC 302, the message packet 408 
requires the return address to be that of the source PC 302. The product information is then 
passed directly to PC 302 over Path "D " 

Referring now to FIGURE 5, the method for detecting and obtaining product information, 
is as follows. In decision block 500, a proprietary application running resident on a source 
computer PC 302 (similar to PC 204) monitors the audio input for a special trigger signal. Upon 
detection of the trigger signal, data following the trigger signal is decoded for further processing, 
in function block 502. In function block 504, the data is buffered for further manipulation. In 
decision block 506, a determination is made as to whether the data can be properly authenticated. 
If not, program flow continues through the "N" signal to function block 520 where the data is 
discarded. In function block 522, the program then signals for a retransmission of the data. The 
system then waits for the next trigger signal, in decision block 500. If properly authenticated in 
decision block 506, program flow continues through the "Y" signal path where the data is then 
used to launch the web browser application, as indicated in function block 508. In function block 
510, the web browser receives the URL data, which is then automatically routed through the 
computer modem 208 to the network interface 304 and ultimately to the network 306. In 
function block 5 14, the ARS 308 responds by returning the URL of advertiser server 3 12 to the 
PC 302. In function block 516, the web browser running on the source PC 302, receives the 
advertiser URL information from the ARS 308, and transmits the URL for the product file to the 
advertiser server 312. In block 5 1 8, the advertiser server 3 1 2 responds by sending the product 
information to the source PC 302 for processing. 

The user may obtain the benefits of this architecture by simply downloading the 
proprietary software over the network. Other methods for obtaining the software are well- 
known; for example, by CD, diskette, or pre-loaded hard drives. 

Referring now to FIGURE 6, there is illustrated a flowchart of the process the ARS 308 
may undergo when receiving the message packet 400 from the source PC 302. In decision block 
600, the ARS 308 checks for the receipt of the message packet 400. If a message packet 400 is 
not received, program flow moves along the "N" path to continue waiting for the message. If the 
message packet 400 is received, program flow continues along path "Y" for message processing. 
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Upon receipt of the message packet 400, in function block 602, the ARS 308 decodes the 
message packet 400. The product code is then extracted independently in function block 604 in 
preparation for matching the product code with the appropriate advertiser server address located 
in the database 310. In function block 606, the product code is then used with a look-up table to 
retrieve the advertiser server 312 URL of the respective product information contained in the 
audio signal data. In function block 608, the ARS 308 then assembles message packet 402 for 
transmission back to the source PC 302. Function block 610 indicates the process of sending the 
message packet 402 back to the source PC 302 over Path "B." 

Referring now to FIGURE 7, there is illustrated a flowchart of the interactive processes 
between the source PC 302 and the advertiser server 312. In function block 700, the source PC 
302 receives the message packet 402 back from the ARS 308 and begins to decode the packet 
402. In function block 702, the URL of the advertiser product information is extracted from the 
message packet 402 and saved for insertion into the message packet 404 to the advertiser server 
312. The message packet 404 is then assembled and sent by the source PC 302 over Path "C" to 
the advertiser server 312, n function block 704. While the source PC 302 waits, in function block 
706, the advertiser server 312 receives the message packet 404 from the source PC 302, in 
function block 708, and disassembles it. The product information location is then extracted from 
the message packet 404 in function block 710. The particular product information is retrieved 
from the advertiser server 3 12 database for transmission back to the source PC 302. In function 
block 712, the product information is assembled into message packet 406 and then transmitted 
back to the source PC 302 over Path "D." Returning to the source PC 302 in function block 714, 
the advertiser product information contained in the message packet 406 received from the 
advertiser server 312, is then extracted and processed in function block 716. 

Referring now to FIGURE 8, after receipt of a trigger signal, a web browser application 
on a source PC 302 is automatically launched and computer display 800 presents a browser page 
802. Proprietary software running on the source PC 302 processes the audio signal data after 
being digitized through the sound card 206. The software appropriately prepares the data for 
insertion directly into the web browser by extracting the product information code and appending 
keystroke data to this information. First, a URL page 804 is opened in response to a Ctrl-0 
command added by the proprietary software as the first character string. Opening URL page 804 
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automatically positions the cursor in a field 806 where additional keystroke data following the 
Ctrl-0 command will be inserted. After URL page 804 is opened, the hypertext protocol 
preamhle http:// is inserted into the field 806. Next, URL information associated with the 
location of the ARS 308 is inserted into field 806. Following the ARS 308 URL data are the 
characters /? to allow entry of variables immediately following the 11 characters. In this 
embodiment, the variable following is the product information code received in the audio signal. 
The product code information also provides the cross-reference information for obtaining the 
advertiser URL from the ARS database 310. Next, a carriage return is added to send the 
URL/product data and close the window 804. After the message packet 400 is transmitted to the 
ARS 308 from the source PC 302, transactions from the ARS308, to the source PC 302, to the 
advertiser server 312, and back to the source PC 302, occur quickly and are transparent to the 
viewer. At this point, the next information the viewer sees is the product information which was 
received from the advertiser server 312. 

Referring now to FIGURE 9, there is illustrated a block diagram of a more simplified 
embodiment. In this embodiment, a video source 902 is provided which is operable to provide an 
audio output on an audio cable 901 which provides routing information referred to by reference 
numeral 904. The routing information 904 is basically information contained within the audio 
signal. This is an encoded or embedded signal. The important aspect of the routing information 
904 is that it is automatically output in realtime as a function of the broadcast of the video 
program received over the video source 902, Therefore, whenever the program is being 
broadcast in realtime to the user 908, the routing information 904 will be output whenever the 
producer of the video desires it to be produced. It should be understood that the box 902 
representing the video source could be any type of media that will result in the routing 
information being output. This could be a cassette player, a DVD player, an audio cassette, a CD 
ROM or any such media. It is only important that this is a program that the producer develops 
which the user 908 watches in a continuous or a streaming manner. Embedded within that 
program, at a desired point selected by the producer, the routing information 904 is output. 

The audio information is then routed to a PC 906, which is similar to the PC 1 12 in 
FIGURE 1 . A user 908 is interfaced with the PC to receive information thereof, the PC 906 
having associated therewith a display (not shown). The PC 906 is interfaced with a network 910, 
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similar to the network 306 in FIGURE 3. This network 910 has multiple nodes thereon, one of 
which is the PC 906, and another of which is represented by a network node 912 which 
represents remote information. The object of the present embodiment is to access remote 
information for display to the user 908 by the act of transmitting from the video program in block 
902 the routing information 904. This routing information 904 is utilized to allow the PC 906 _ 
which has a network "browser" running thereon to "fetch" the remote information at the node 
912 over the network 91 0 for display to the user 908. This routing information 904 is in the form 
of an embedded code within the audio signal, as was described hereinabove. 

Referring now to FIGURE 10, there is illustrated a more detailed block diagram of the 
embodiment of FIGURE 9. In this embodiment, the PC 906 is split up into a couple of nodes, a 
first PC 1002 and a second PC 1004. The PC 1002 resides at the node associated with the user 
908, and the PC 1 004 resides at another node. The PC 1004 represents the ARS 308 of FIGURE 
3. The PC 1004 has a database 1 006 associated therewith, which is basically the advertiser 
database 310. Therefore, there are three nodes on the network 910 necessary to implement the 
disclosed embodiment, the PC 1002, the PC 1004 and the remote information node 912. The 
routing information 904 is utilized by the PC 1002 for routing to the PC 1004 to determine the 
location of the remote information node 912 on the network 910. This is returned to the PC 1002 
and a connection made directly with the remote information node 912 and the information 
retrieved therefrom to the user 908. The routing information 904 basically constitutes primary 
routing information. 

Referring now to FIGURE 1 1, there is illustrated a diagrammatic view of how the 
network packet is formed for sending the primary routing information to the PC 1 004. In 
general, the primary routing information occupies a single field which primary routing 
information is then assembled into a data packet with the secondary routing information for 
transfer to the network 910. This is described hereinabove in detail. 

Referring now to FIGURE 12, there is illustrated an alternate embodiment to that of 
FIGURE 9. In this embodiment, the video source 902 has associated therewith an optical region 
1202, which optical region 1202 has disposed therein an embedded video code. This embedded 
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video code could be relatively complex or as simple as a grid of dark and white regions, each 
region in the grid able to have a dark color for a logic "1" or a white region for a logic "0." This 
will allow a digital value to be disposed within the optical region 1202. A sensor 1204 can then 
be provided for sensing this video code. In the example above, this would merely require an 
array of optical detectors, one for each region in the grid to determine whether this is a logic "1" _ 
or a logic "0" state. One of the sensed video is then output to the PC 906 for processing thereof 
to determine the information contained therein, which information contained therein constitutes 
the primary routing information 904. Thereafter, it is processed as described hereinabove with 
reference to FIGURE 9. 

Referring now to FIGURE 13, there is illustrated a block diagram for an embodiment 
wherein a user's profile can be forwarded to the original subscriber or manufacturer. The PC 906 
has associated therewith a profile database 1302, which profile database 1302 is operable to store 
a profile of the user 908. This profile is created when the program, after initial installation, 
requests profile information to be input in order to activate the program. In addition to the 
profile, there is also a unique ID that is provided to the user 908 in association with the browser 
program that runs on the PC 906. This is stored in a storage location represented by a block 
1304. This ID 1304 is accessible by a remote location as a "cookie" which is information that is 
stored in the PC 906 in an accessible location, which accessible location is actually accessible by 
the remote program running on a remote node. 

The ARS 308, which basically constitutes the PC 1004 of FIGURE 1 0, is operable to 
have associated therewith a profile database 1308, which profile database 1308 is operable to 
store profiles for all of the users. The profile database 1 308 is a combination of the stored in 
profile database 1302 for all of the PCs 906 that are attachable to the system. This is to be 
distinguished from information stored in the database 3 1 0, the advertiser's database, which 
contains intermediate destination tables. When the routing information in the primary routing 
information 904 is forwarded to the ARS 308 and extracted from the original data packet, the 
look-up procedure described hereinabove can then be performed to determine where this 
information is to be routed. The profile database 1302 is then utilized for each transaction, 
wherein each transaction in the form of the routing information received from the primary routing 
information 904 is compared to the destination tables 310 to determine what manufacturer it is 
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associated with. The associated ID 1304 that is transmitted along with the routing information in 
primary routing information 904 is then compared with the profile database 1308 to determine if 
a profile associated therewith is available. This information is stored in a transaction database 
1310 such that, at a later time, for each routing code received in the form of the information in 
primary routing information 904, there will associated therewith the IDs 1304 of each of the PCs _ 
906. The associated profiles in database 1308, which are stored in association with IDs 1304, can 
then be assembled and transmitted to a subscriber as referenced by a subscriber node 1 31 2 on the 
network 910. The ARS 308 can do this in two modes, a realtime mode or a non-realtime mode. 
In a realtime mode, each time a PC 906 accesses the advertiser database 310, that user's profile 
information is uploaded to the subscriber node 1312. At the same time, billing information is 
generated for that subscriber 1312 which is stored in a billing database 1316. Therefore, the ARS 
308 has the ability to inform the subscriber 13 12 of each transaction, bill for those transactions, 
and also provide to the subscriber 1312 profile information regarding who is accessing the 
particular product advertisement having associated therewith the routing information field 904 
for a particular routing code as described hereinabove. This information, once assembled, can 
then be transmitted to the subscriber 1312 and also be reflected in billing information and stored 
in the billing information database 1316. 

Referring now to FIGURE 14, there is illustrated a flowchart depicting the operation for 
storing the profile for the user. The program is initiated in a block 1402 and then proceeds to a 
function block 1404, wherein the system will prompt for the profile upon initiation of the system. 
This initiation is a function that is set to activate whenever the user initially loads the software 
that he or she is provided. The purpose for this is to create, in addition to the setup information, a 
user profile. Once the user is prompted for this, then the program will flow to a decision block 
1406 to determine whether the user provides basic or detailed information. This is selectable by 
the user. If selecting basic, the program will flow to a function block 1408 wherein the user will 
enter basic information such as name and serial number and possibly an address. However, to 
provide some incentive to the user to enter more information, the original prompt in function 
block 1404 would have offers for such things as coupons, discounts, etc, if the user will enter 
additional information. If the user selects this option, the program from the decision block 1406 
to a function block 1410. In the function block 1410, the user is prompted to enter specific 
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information such as job, income level, general family history, demographic information and 
more. There can be any amount of information collected in this particular function block. 

Once all of the information is collected, in either the basic mode or the more specific 
mode, the program will then flow to a function block 1412 where this information is stored 
locally. The program then flows to a decision block 1414 to then go on-line to the host or the 
ARS 308. In general, the user is prompted to determine whether he or she wants to send this 
information to the host at the present time or to send it later. If he or she selects the "later" 
option, the program will flow to a function block 141 5 to prompt the user at a later time to send 
the information. In the disclosed embodiment, the user will not be able to utilize the software 
until the profile information is sent to the host. Therefore, the user may have to activate this at a 
later time in order to connect with the host. 

If the user has selected the option to upload the profile information to the host, the 
program will flow to the function block 1416 to initiate the connect process and then to a 
decision block 1418 to determine if the connection has been made. If not, the program will flow 
along a "N" path to a time to decision block 1420 which will time to an error block 1422 or back 
to the input of the connect decision block 1418. The program, once connected, will then flow 
along a "Y" path from decision block 141 8 to a function block 1428 to send the profile 
information with the ID of the computer or user to the host. The ID is basically, as described 
hereinabove, a "cookie" in the computer which is accessed by the program when transmitting to 
the host. The program will then flow to a function block 1430 to activate the program such that 
it, at later time, can operate without requiring all of the set up information. In general, all of the 
operation of this flowchart is performed with a "wizard" which steps the user through the setup 
process. Once complete, the program will flow to a Done block 1432. 

Referring now to FIGURE 1 5, there is illustrated a flowchart depicting the operation of 
the host when receiving a transaction. The program is initiated at a start block 1 502 and then 
proceeds to decision block 1504, wherein it is determined whether the system has received a 
routing request, i.e., the routing information 904 in the form of a tone, etc., embedded in the 
audio signal as described hereinabove with respect to FIGURE 9. The program will loop back 
around to the input of decision block 1504 until the routing request has been received. At this 
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time, the program will flow along the "Y" path to a function block 1506 to receive the primary 
routing information and the user ID. Essentially, this primary routing information is extracted ' 
from the audio tone, in addition to the user ID. The program then flows to a function block 1508 
to look up the manufacturer URL that corresponds to the received primary routing information 
and then return the necessary command information to the originating PC 108 in order to allow ^ 
that PC to connect to the destination associated with the primary routing information. Thereafter, 
the program will flow to a function block 1510 to update the transaction database 1310 for the 
current transaction. In general, the routing information 904 will be stored as a single field with 
the associated IDs. The profile database, as described hereinabove, has associated therewith 
detailed profiles of each user on the system that has activated their software in association with 
their ID. Since the ID was sent in association with the routing information, what is stored in the 
transaction database is the routing code, in association with all of the IDs transmitted to the 
system in association with that particular routing code. Once this transaction database has been 
updated, as described hereinabove, the transactions can be transferred back to the subscriber at 
node 312 with the detailed profile information from the profile database 1308. 

The profile information can be transmitted back to the subscriber or manufacturer in the 
node 312 in realtime or non-realtime. A decision block 1512 is provided for this, which 
determines if the delivery is realtime. If realtime, the program will flow along a "Y" path to a 
function block 1514 wherein the information will be immediately forwarded to the manufacturer 
or subscriber. The program will then flow to a function block 1516 wherein the billing for that 
particular manufacturer or subscriber will be updated in the billing database 1316. The program 
will then flow into an End block 1518. If it was non-realtime, the program moves along the "N" 
path to a function block 1520 wherein it is set for a later delivery and it is accrued in the 
transaction database. In any event, the transaction database will accrue all information associated 
with a particular routing code. 

With a realtime transaction, it is possible for a manufacturer to place an ad in a magazine 
or to place a product on a shelf at a particular time. The manufacturer can thereafter monitor the 
times when either the ads are or the products are purchased. Of course, they must be scanned 
into a computer which will provide some delay. However, the manufacturer can gain a very 
current view of how a product is moving. For example, if a cola manufacturer were lo provide a 



WO 00/16205 PCT/IJS99/20963 

20 

promotional advertisement on, for example, television, indicating that a new cola was going to be 
placed on the shelf and that the first 1000 purchasers, for example, scanning their code into the " 
network would receive some benefit, such as a chance to win a trip to some famous resort in 
Florida or some other incentive, the manufacturer would have a very good idea as to how well the 
advertisement was received. Further, the advertiser would know where the receptive markets 
were. If this advertiser, for example, had placed the television advertisement in ten cities and 
received overwhelming response from one city, but very poor response from another city, he 
would then have some inclination to believe that either one poor response city was not a good 
market or that the advertising medium he had chosen was very poor. Since the advertiser can 
obtain a relatively instant response and also content with that response as to the demographics of 
the responded very important information can be obtained in a relatively short time. 

It should be noted that the disclosed embodiment is not limited to a single source PC 302, 
but may encompass a large number of source computers connected over a global communication 
network. Additionally, the embodiment is not limited to a single ARS 308 or a single advertiser 
server 3 12, but may include a plurality of ARS and advertiser systems, indicated by the addition 
of ARS 314 and advertiser server A 316, respectively. It should also be noted that this 
embodiment is not limited only to global communication networks, but also may be used with 
LAN, WAN, and peer-to-peer configurations. 

It should also be noted that the disclosed embodiment is not limited to a personal 
computer, but is also applicable to, for example, a Network Computer ("NctPC"), a scaled-down 
version of the PC, or any system which accommodates user interaction and interfaces to 
information resources. 

One typical application of the above noted technique is for providing a triggering event 
during a program, such as a sport event. In a first example, this may be generated by an 
advertiser. One could imagine that, due to the cost of advertisements in a high profile sports 
program, there is a desire to utilize this time widely. If, for example, an advertiser contracted for 
15 seconds worth of advertising time, they could insert within their program a tone containing the 
routing information. This routing information can then be output to the user's PC which will 
cause the user's PC to, via the network, obtain information from a remote location typically 
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controlled by the advertiser. This could be in the form of an advertisement of a length longer 
than that contracted for. Further, this could be an interactive type of advertisement. An 
important aspect to the type of interaction between the actual broadcast program with the 
embedded routing information and the manufacturer's site is the fact that there is provided in the 
information as to the user's PC and a profile of the user themselves. Therefore, an advertiser can. 
actually gain realtime information as to the number of individuals that are watching their 
particular advertisement and also information as to the background of those individuals, 
demographic information, etc. This can be a very valuable asset to an advertiser. 

In another example, the producer of the program, whether it be an on-air program, a 
program embedded in a video tape, CD-ROM, DVD. or a cassette, can allow the user to 
automatically access additional information that is not displayed on the screen. For example, in a 
sporting event, various statistics can be provided to the user from a remote location, merely by 
the viewer watching the program. When these statistics are provided, the advertiser can be 
provided with demographic information and background information regarding the user. This 
can be important when, for example, the user may record a sports program. If the manufacturer 
sees that this program routing code is being output from some device at a later time than the 
actual broadcast itself, this allows the advertisers to actually see that their program is still being 
used and also what type of individual is using it. Alternatively, the broadcaster could determine 
the same and actually bill the advertiser an additional sum for a later broadcast. This is all due to 
the fact that the routing information automatically, through a PC and a network, will provide an 
indication to the advertiser for other intermediary regarding the time at which the actual 
information was broadcast. 

The different type of medium that can be utilized with the above embodiment are such 
things as advertisements, which are discussed hereinabove, contests, games, news programs, 
education, coupon promotional programs, demonstration media (demos), photographs, all of 
which can be broadcast on a private site or a public site. This all will provide the ability to allow 
realtime interface with the network and the remote location for obtaining the routed information 
and also allow for realtime billing and accounting. 
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Although the preferred embodiment has been described in detail, it should be understood 
that various changes, substitutions and alterations can be made therein without departing from the 
spirit and scope of the invention as defined by the appended claims. 
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1 . A method for controlling a computer, comprising the steps of: 

inputting an analog signal at a source location on a network, said analog signal 
comprising a product identifier which is in close association with a product such that it identifies ... 
the associated product; 

appending routing information to the product identifier, which routing information 
defines the location of a remote location on the network; and 

transmitting the product identifier from the source location to the remote location 
over the network in accordance with the appended routing information, which routing 
information defines the network connection between the source location and the remote location 
on the network, wherein information regarding the associated product can be returned to the 
source location from the remote location. 

2. The method of Claim 1 , wherein the step of transmitting the product information 
is performed in response to the step of inputting. 

3. The method of Claim 1, wherein the step of appending the routing information 
occurs in response to the step of inputting. 



4. 



The method of Claim 1, wherein the analog signal is input to a sound card. 
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5. The method of Claim 1, wherein the routing information is associated with a 
secondary location on the network such that the step of transmitting to the remote location 
transmits the product identifier to the remote location in association with routing information, for 
returning to the source location, and further comprising: 

accessing a database of product routing information which provides an association, 
between a predetermined product identifier and a remote producl information location on the 
network, there being a plurality of such product routing information stored in the database; 

accessing the database in response to receiving at the secondary location a 
transmitted product identifier from the source location; 

comparing the received product identifier with the stored product routing 
information in the database; 

if there is a match between the received product identifier and any of the stored 
product routing information, transmitting the matching product routing information back to the 
source location; and 

at the source location, in response to receiving the matching product routing 
information, interconnecting the source location with the product information location over the 
network and receiving product information therefrom. 

6. The method of Claim 5, and further comprising the step of displaying the received 
product information. 

7. The method of Claim 1 , wherein the network is a global communication network 
that provides a universal resource locator (URL) for each location on the network and the routing 
information is comprised of the URL for the location. 

8. The method of Claim 7, wherein a processing system is provided running a 
program that receives input in the form of keystrokes for routing of information thereto, and the 
step of appending routing information comprises the step of forming a keystroke string that 
emulates the combination of the routing information and the product identifier in keystrokes and 
transmits the emulated keystrokes to the program in the same manner that a user would key the 
strokes in, and the step of transmitting is performed in response to the appended routing 
information and product identifier transmitted to the program with an appended command 
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associated therewith that instructs the program to perform a transmit operation in a manner 
similar to a user interacting with the program to transmit routing information. 

9. The method of Claim 8, wherein the appended routing information includes 
instructional information as to how the remote location is to handle the transmitted product 
identifier. 



10. The method of Claim 9, wherein the program is a web browser which is 
automatically launched in response to said analog signal. 
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11. A method for retrieving information from a storage region having a defined 
location, comprising the steps of: 

broadcasting a program having embedded therein a routing signal having routing 
information contained therein; 

extracting the routing signal from the broadcast; and 

controlling a personal computer for a user to retrieve the information from a 
storage region at the defined location, which defined location is located with the extracted routing 
information, providing it at the personal computer for use by the user. 

12. The method of Claim 1 1 , wherein the personal computer has a display associated 
therewith, wherein the retrieved information is displayed on the display. 

13. The method of Claim 11, wherein the broadcast program is a video program. 

14. The method of Claim 11, wherein the broadcast program is an audio program. 

15. The method of Claim 11, wherein the routing signal comprises an audio signal and 
wherein the step of extracting comprises decoding the information in the audio signal by the 
personal computer. 

16. The method of Claim 15, wherein the broadcast program is a video program. 

17. The method of Claim 15, wherein the broadcast program is an audio program. 

18. The method of Claim 15, wherein the step of extracting routing information 
comprises connecting the video source to the personal computer with a cable which carries only 
audio information, the personal computer then decoding from the audio information in the step of 
extracting the routing information from the routing signal. 

19. The method of Claim 11, wherein the routing signal is a video signal and the 
broadcast program is a video program and the step of extracting comprises extracting the 
embedded video information from the video routing signal. 
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20. The method of Claim 11, wherein the storage region is disposed on a remote 
location on a network, and the step of controlling comprises: 

transferring the routing information to an intermediate node on the network; 

storing at the intermediate node on the network lookup information defining the 
location of the storage region on the network correlated with the routing information, such that a ^ 
unique routing information is associated with a particular storage region on the network; 

looking up at the intermediate node the correlated location of the storage region 
returning the location information to the personal computer; 

accessing the information from the location information retrieved from the 
intermediate node and the storage base associated therewith and retrieving the information 
therefrom for delivery to the personal computer and for use by the user. 

21 . The method of Claim 20 and further comprising the step of storing at a desired 
location profile information associated with the user and storing at the personal computer 
identification information as associated with the profile information at the desired location and 
further comprising the step of transmitting the identification information to the intermediate node 
when transmitting the routing information thereto, which identification information can be stored 
thereat for later association with the stored profile information at the desired location. 

22. The method of Claim 2 1 , and further comprising the step of transmitting profile 
information from the desired location to a remote subscriber location which remote subscriber 
location is associated with the routing information, such that profile information is transmitted to 
the routing information from the user and the associated identification signals that is associated 
with the extracted routing information from the broadcast. 

23. The method of Claim 22, wherein the identification signals are accumulated for 
each routing signal transmitted from different locations on the network and different personal 
locations on the network and different personal computers which can be transmitted to the 
subscriber location at a later time. 



WO 00/16205 



PCT/US99/20963 



ADVERTISER 
DATA INPUT 



AUDIO MODULATE 



TV PROGRAM 
SOURCE 

1 

104 



109- 



FIG. 2 



1/6 



-100 
-102 



101 



108 



117 



106 




1- 



109- 



FIG. 



110 


DISPLAY 




118 

114 

t 








PC 




MODEM _ 


113 






112 1 


^ 116-^ 


PROGRAM 
INFORMATION 


TO 
NETWORK 



204 



110 

} 






^-206 








DISPLAY y 


RECEIVER 






SOUND 
CARD 


CPU 






ADVERTISER 
WEB PAGE 
212 












MODEM 






\ 

TO 




^■208 


oo 











NETWORK 




! ADVERTISER I 
I DATABASE J 



WO 00/16205 



2/6 



PCT/US99/20963 



PATH A: SOURCE TO ARS 



400— 


URL OF 
ARS 


ADVERTISER 
PRODUCT CODE 


URL OF 
SOURCE 






FIG. 4 a 




PATH B: ARS TO SOURCE 






402— 


URL OF 
SOURCE 


URL OF 
ADVERTISER 


URL OF 
ARS 






FIG. 4 b 




PATH C: SOURCE TO ADVERTISER 




404^ 


URL OF 
ADVERTISER 


REQUEST-FQR- 
IN FORMATION 


URL OF 
SOURCE 






FIG. 4c 




PATH D: 


ADVERTISER TO SOURCE 




406— 


URL OF 
SOURCE 


REQUESTED 
INFORMATION 


URL OF 
ADVERTISER 






FIG. 4d 




PATH E: ARS TO ADVERTISER (OPTIONAL) 




408— 


URL OF 
ADVERTISER 


REQUEST FOR 
INFORMATION 


URL OF 

SOURCE 



FIG. 4e 



602- 
604- 



C START ) 




DECODE PACKET 



EXTRACT PRODUCT CODE 



LOOK-UP URL OF 

ADVERTISER 
BASED ON CODE 

I 



ASSEMBLE PACKET 

I 



FIG. 6 



SEND TO SOURCE 
(PATH B) 

C END ) 



-606 



-608 



-610 



WO 00/16205 



3/6 



PCT/US99/20963 



( START ) 




DECODE THE 
FOLLOWING DATA 



I 



BUFFER THE DATA 




LAUNCH WEB BROWSER 



INSERT URL DATA 
INTO WEB BROWSER 



SEND URL TO 
GLOBAL NETWORK 



ARS SERVER RESPONDS 
WITH URL OF 
ADVERTISER SERVER 



PC TRANSMITS REQUEST 
FOR PRODUCT INFORMATION 
TO ADVERTISER SERVER 



ADVERTISER SERVER 

SENDS PRODUCT 
INFORMATION TO PC 



C START ) 



522 



SIGNAL FOR 
RETRANWISSION 

t 



RECEIVE ARS PACKET 

I 



EXTRACT URL OF 
ADVERTISER PRODUCT 
INFORMATION 



DISCARD 
520 



SEND MESSAGE TO 

ADVERTISER 
SERVER (PATT1 C) 



WAIT 



-700 



-702 



-704 



-706 



-508 



-510 



-512 



-514 



-516 



-518 



RECEIVE SOURCE 
PACKET 



EXTRACT PRODUCT 
INFORMATION 
FILE LOCATION 



J. 



TRANSMIT PRODUCT 
INFORMATION TO 
SOURCE (PATH D) 



-708 



-710 



-712 



ADVERTISER 

SERVER 



RETURN 



PROCESS ADVERTISER 
INFORMATION 



-714 



-716 



I 



( END ) 

FIG. 5 



( END ) 

FIG. 7 



WO 00/16205 



4/6 



PCT/US99/20963 



800- 



D1SPLAY 



804 



BROWSER PAGE 



806 
±1 



http://URL[/?][ADVERTISER DATA] 



802 



o o 



808 



FIG. 8 



URL/ADVERTISER DATA 



[Ctrl-0] [htlp://] [URL] [/?] [variable] [ J ] 



OPEN PAGE 

USE HYPERTEXT PROTOCOL- 

URL OF ARS 

ALLOW VARIABLE APPENDING 

APPENDED VARIABLE (e.g., UPC CODE)- 
CARRIAGE RETURN - TRANSMIT DATA- 



902 
VI DEO 



901 
AUDIO / . 



904^ (ROUTING 
INFORMATION) 



USER 



906 

s 


910 

) 


912 


PC 




NET 




REMOTE 






INFORMATION 



-908 



FIG. 9 



WO 00/16205 



5/6 



PCT/US99/20963 



902- 



VI DEO 



901 

/ AUDIO 



904 -v. (ROUTING 
INFORMATION) 



FIG. 1 0 



908- 



1002 

V 


910 

7 


PC 




NET 




1 






USER 




REMOTE 
INFORMATION 



912 



1004 

I 



PC 



DATABASE 



-1006 



VIDEO SOURCE 



1204 



OPTICAL REGION 









902 



1202 



SENSED VIDEO 



FIG. 12 



PRIMARY ROUTING 
INFORMATION 


FIG. 1 1 








PRIMARY ROUTING 
INFORMATION 


SECONDARY ROUTING 
INFORMATION 



908- 



USER 



902 1302- 

\ 



VIDEO 









PROFILE 
DATABASE 




ID 



-1304 



901 

/ AUDIO 



904 -x. (ROUTING 

INFORMATION) ggg. 



908- 



PC 










USER 


31?^ 



910 

s 



NET 



ARS 



FIG. 1 3 

310 



INTERMEDIATE 
DESTINATION 
TABLES _ 



PROFILE 
DATABASE 



1308 



906 

S 


910 

/ 


912 

* 


PC 




NET 




REMOTE 






INFORMATION 



SUBSCRIBER 



REMOTE 
INFORMATION 



BILLING 



TRANSACTION 



-1312 
-912 

-1316 
-1310 



WO 00/16205 



6/6 



PCT/US99/20963 



QwrTV -1402 



PROMPT FOR PROFILE 



1406 



1410 




-1404 



BASIC? 
NOT 



YES 



_ 1408 
1 j 



1502-h : start ) 
1504 



ENTER BASIC 
INFORMATION 



ENTER SPECIFIC 
INFORMATION, JOBS, 
INCOME, HISTORY, 
DEMOGRAPHICS, ETC. 




STORE LOCALLY 



-1412 



1414 




1506- 



1508- 



1510- 



RX PRIMARY ROUTING 
INFORMATION 
AND USER ID 



LOOK UP MANUFACTURER'S 
URL AND RETURN 



UPDATE TRANSACTION 

DATABASE FOR 
CURRENT TRANSACTION 




1420 



SET FOR 

LATER 
DELIVERY 



SEND PROFILE 
INFORMATION WITH ID 



U99- ^ ERROR ) 
-1428 



-1520 
1516- 





1514 






FORWARD TO 


MANUFACTURER 



UPDATE 
BILLING 



1518 ^<_END_J 

FIG. 15 



TERMINATE 
AND ACTIVATE 



-1430 



( DONE K 1432 

FIG. 14 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US99/20963 



A. CLASSIFICATION OF SUBJECT MATTER 
1PC(6) :G06F 13/00, 15/16 

US CL : 709/238, 239, 249 
According to International Patent Classification (IPC) or to both national classification and IPC 

B. FIELDS SEARCHED 

Minimum documentation searched (classification system followed by classification symbols) 

U.S. : 709/238 , 239 , 249 , 218 , 234; 345/327 

Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search ( 
WEST, EAST 


name of data base and, where practicable, search terms used) 


C. DOCUMENTS CONSIDERED TO BE RELEVANT 


Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


X,P 


US 5,854,897 A (RADZIEWICZ et al) 29 December 1998, coL 7, 
lines 18-67, col. 9, line 64 - coL 10, line 52, col. 11, lines 6-29, 
col. 12, lines 23-45. 


1-23 


A 


US 5,774,664 A (HIDARY et al) 30 June 1998, col. 2, line 33 - col. 
3, line 18, col. 4, lines 28-39. 


1-23 


A,P 


US 5,929,850 A (BROADWIN et al) 27 July 1999, col. 1, lines 31- 
58, col. 4, line 65 - coL 5, line 8, lines 34-42, col. 17, line 61 - col. 
18, line 55. 


1-23 


A,E 


US 5,974,451 A (SIMMONS) 26 October 1999, col. line 27 - col. 
2, line 10, col. 7, lines 6-65. 


1-3, 11, 20-23 


| { Further documents are listed in the continuation of Box C. { ^ | Sec patent family annex. 


* Special citajorwi of cilad document* 

*A' document defining the general i tale of the art which is not considered 
to be of particular relevance 


"T" later document published after the international filing date or priority 
date and not in conflict with the application but cited to understand 
the principle or theory underlying the invention 

'X" document of particular relevance; the claimed invention cannot be 


"L* document which may throw doubts on priority claun(i) or which is 
chad to establish the publication date of another citation or other 
special reason (as specified) 

*0* document referring to an oral disclosure, use, exhibition or other 

■P" document published prior u> the international filing date but later than 
the priority dale claimed 


co-naidered nor el or cannot ba considered to involve an inventive step 
when the document is taken alone 

'Y* document of particular relevance; the claimed invention cannot be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the an 

*&.' document member of the same patent family 


Date of the actual completion of the international search 
20 NOVEMBER 1999 


Date of mailing of the international search report 

0£dec jggq 


Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D.C. 20231 
Facsimile No. (703) 305-3230 


Authorized officer 

FRANKJASTA % /tfdfitfZicD 
Telephone No. (703) 305-3817 ( 



Form PCT/ISA/210 (second sheet)(July 1992)* 



